Combining a Mixture of Experts with Transfer Learning in Complex Games

We present a supervised approach for learning policies in a highly complex game from small amounts of human data consisting of state–action pairs. Our Neural Network architecture can adapt to the varying size of the set of legal actions, thus bypassing the need to hardcode the actions in the output layer or iterate over them. This makes the training more data efficient. We use synthetic data created via game simulations among AI agents to show that a mixture of experts, where each expert predicts actions in different portions of the game, improves performance. We then show that this approach applied to human data also improves performance: in particular, using transfer learning to enable one expert to learn from another enhances performance on those portions of the game for which there is relatively little training data compared to other portions. The domain chosen for evaluation is the board game Settlers of Catan.